Asymptotics of Posteriors and Model Selection

CHAPTER 4. ASYMPTOTICS OF POSTERIORS AND MODEL SELECTION 4.1 Consistency of posteriors. Given a measurable family {P θ , θ ∈ Θ}, dominated by a σ-finite measure v, for a measurable space (Θ, T), a prior π on Θ, and observations X 1 , X 2 ,. .. i.i.d. (P θ 0) for some θ 0 ∈ Θ, we have posteriors π x,n on Θ where x = (X 1 ,. .. , X n) for each n. Recall that we have defined the posteriors by multiplying the prior by the likelihood function Π n j=1 f (θ, X j) and normalizing the result, if possible, to be a probability measure (Proposition 1.3.5). (For Theorem 4.1.4 below, where {P θ , θ ∈ Θ} is not necessarily dominated, a more general definition of posteriors will be used.) The posteriors will be called consistent if for every neighborhood U of θ 0 , π x,n (U) → 1 almost surely as n → ∞. This form of consistency is not for estimators T n , but is just a property of the prior and the likelihood function. In some situations, consistency of posteriors can lead to consistency of estimators. For example, if Θ is an interval in R, and T n is a median of the posterior law π x,n , then consistency of posteriors will imply that T n are consistent. If the interval is bounded, T n could also be taken as the mean of π x,n. If the prior π has π(U) = 0 for some neighborhood U of the true parameter θ 0 , then π x,n (U) = 0 for all x and n, so the posteriors can't be consistent. On the other hand, if π(U) > 0 for every neighborhood U of θ 0 , then under some conditions as in Section 3.3, it will be shown that the posteriors are consistent. It can happen in pathological cases that the posteriors are not consistent, for example if as the neighborhoods U shrink to {θ 0 }, π(U) → 0 very fast, and if the likelihood function doesn't behave well. Such an example will be given in Proposition 4.1.2 and after it. 4.1.1 Theorem. Assume that: (i) {P θ , θ ∈ Θ} is a measurable family, dominated by a σ-finite measure v, and identifiable, so that P θ = P φ for θ = φ; …